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Abstract 

We study in this paper the problem of adaptive trajectory tracking for nonlinear systems affine in the control 
with bounded state-dependent and time-dependent uncertainties. We propose to use a modular approach, in 
the sense that we first design a robust nonlinear state feedback which renders the closed loop input to state 
stable (ISS) between an estimation error of the uncertain parameters and an output tracking error. Next, we 
complement this robust ISS controller with a model-free multiparametric extremum seeking (MES) algorithm 
to estimate the model uncertainties. The combination of the ISS feedback and the MES algorithm gives an 
indirect adaptive controller. We show the efficiency of this approach on a two-link robot manipulator example. 

1 Introduction 

Input-output feedback linearization has been proven to be a powerful control design for trajectory tracking and 
stabilization of nonlinear systems |T. The basic idea is to first transform a nonlinear system into a simplified 
linear equivalent system and then use the linear design techniques to design controllers in order to satisfy stability 
and performance requirements. One shortcoming of the feedback linearization approach is that it requires precise 
system modelling [lj. When there exist model uncertainties, a robust input-output linearization approach needs 
to be developed. For instance, high-gain observers [2] and linear robust controllers have been proposed in 
combination with the feedback linearization techniques. Another approach to deal with model uncertainties is 
using adaptive control methods. Of particular interest to us is the modular approach to adaptive nonlinear 
control, e.g. [1]. In this approach, first the controller is designed by assuming all the parameters are known and 
then an identifier is used to guarantee certain boundedness of the estimation error. The identifier is independent 
of the designed controller and thus this is called ‘modular’ approach. 
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On the other hand, extremum seeking (ES) method is a model-free control approach, e.g.[5], which has been 
applied to many industrial systems, such as electromagnetic actuators , compressors [8], and stirred-tank 
bioreactors [5] . Many papers have dedicated to analyzing the ES algorithms convergence when applied to a static 
or a dynamic known maps, e.g. mrnmm, however, much fewer papers have been dealing with the use of ES in 
the case of static or dynamic uncertain maps. The case of ES applied to an uncertain static and dynamic mapping, 
was investigated in m, where the authors considered systems with constant parameter uncertainties. However, 
in 1 13] , the authors used ES to optimize a given performance (via optimizing a given performance function), 
and complemented the ES algorithm with classical model-based filters/estimators to estimate the states and the 
unknown constant parameters of the system, which is one of the main differences with the approach that we want 
to present here (originally introduced by the authors for a specific mechatronics application in [7J [TTI, IT?] ), where 
the ES is not only used to optimize a given performance but is also used to estimate the uncertainties of the 
model, without the need for extra model-based filters/estimators. 

In this work, we build upon the existing ES results to provide a framework which combines ES results and robust 
model-based nonlinear control to propose an ES-based indirect adaptive controller, where the ES algorithm is 
used to estimate, in closed-loop, the uncertainties of the model. Furthermore, we focus here on a particular 
class of nonlinear systems which are input-output linearizable through static state feedback [16]. We assume that 
the uncertainties in the linearized model are bounded additive as guaranteed by the ‘matching condition ’ M- 
The control objective is to achieve asymptotic tracking of a desired trajectory. The proposed adaptive control is 
designed as follows. In the first step, we design a controller for the nominal model (i.e. when the uncertainties 
are assumed to be zero) so that the tracking error dynamics is asymptotically stable. In the second step, we use 
a Lyapunov reconstruction method jTS] to show that the error dynamics are input-to-state stable (ISS) [Hi [H] 
where the estimation error in the parameters is the input to the system and the tracking error represents the 
system state. Finally, we use ES to estimate the uncertain model parameters so that the the tracking error will 
be bounded and decreasing, as guaranteed by the ISS property. To validate the results, we apply our results on 
a two-link robotic manipulators EU. 

Similar ideas of ES-based adaptive control for nonlinear systems have been introduced in ®[7]. In these two 
works, the problem of adaptive robust control of electromagnetic actuators was studied, where ES was used to 
tune the feedback gains of the nonlinear controller in |B] and ES was used to estimate the unknown parameters 
in [7]. An extension to the general case of nonlinear systems was proposed in EH [22]. We relax here the strong 
assumption, used in EDE2], about the existence of an ISS feedback controller, and propose a constructive proof 
to design such an ISS feedback for the particular case of nonlinear systems affine in the control. 

The rest of the paper is organized as follows. In Section [2] we present notations, and some fundamental 
definitions and results that will be needed in the sequel. In Section [3] we provide our problem formulation. 
The nominal controller design are presented in Section 14.21 In Section 14.31 a robust controller is designed which 
guarantees ISS from the estimation errors input to the tracking errors state. In Section 14.71 the ISS controller 
is complemented with an MES algorithm to estimate the model uncertainties. Section [5] is dedicated to an 


application example and the paper conclusion is given in Section [6] 


2 Preliminaries 

Throughout the paper, we use || • || to denote the Euclidean norm; i.e. for a vector x £ R”, we have 
ll^ll — IMI 2 = \/x T x , where x T denotes the transpose of the vector x. The 1-norm of x £ R ra is denoted by 
||x||i. We use the following norm properties for the need of our proof: 

1. for any x £ R”, ||x|| < ||x||i; 


2. for any x, y £ R™, ||x|| - ||y|| < ||x-y||; 

3. for any x, y £ R n , x T y < ||x||||y||. 

Given x £ R m , the signum function is defined as 


sign(x) = [sign(xi), sign(x 2 ), •••, sign(x m )] r , 


where x, denotes the i-th (1 < i < m) element of x and 


sign(xj) 


1 if Xi > 0 

< 0 if Xi = 0 

— 1 if Xi < 0 


We have x T sign(x) = ||x||i. 

For an n x n matrix P, we denote by P > 0 if it is positive definite. Similarly, we denote by P < 0 if it is 
negative definite. We use diag{Ai, A 2 , ■ ■ ■ , A n } to denote a diagonal block matrix with n blocks. For a matrix 
B , we denote as the element that locates at the i-th row and j-tli column of matrix B. We denote I n as 

the identity matrix or simply / if the dimension is clear from the context. 

We use / to denote the time derivative of / and /^(t) for the r-th derivative of fit), i.e. /^ 

We denote by C fc functions that are k times differentiable and by C°° a smooth function. A continuous function 
a : [0, a) —> [0, 00 ) is said to belong to class 1C if it is strictly increasing and a(0) = 0. It is said to belong to class 
/Coo if a = 00 and a(r) —> 00 as r —> 00 [16]. A continuous function /? : [0, a) x [0, 00 ) —► [0, 00 ) is said to belong 
to class ICC if, for a fixed s, the mapping /3(r, s) belongs to class K, with respect to r and, for each fixed r, the 
mapping /3(r, s) is decreasing with respect to s and /3(r, s) —> 0 as s —> 00 [15] . 

Consider the system 


(2.1) x = f(t,x,u) 

where / : [0, 00 ) x R ra x R m —> R" is piecewise continuous in t and locally Lipschitz in x and u, uniformly in t. 
The input u(t) is piecewise continuous, bounded function of t for all t > 0. 




Definition 2.1. C jlBl [25] ) The system \2.1\) is said to be input-to-sate stable (ISS) if there exist a class ICC 
function /3 and a class 1C function 7 such that for any initial state x(to) and any bounded input u[t), the solution 
x(t) exists for all t > to and satisfies 


IK*)II <P(\\x(to)\\,t-t 0 )+'y( sup ||m(t)||). 

to<T<t 

Theorem 2.1. ( jl6l [23]) Let V : [0,00) x R ra —> R be a continuously differentiable function such that 


MINI) <V(t,x) < a 2 (||x||) 

( 2 . 2 ) ?Y- + ^f(t,x,u)<-W(x), V||x|| > p(||u||) > 0 

for all (t,x,u) £ [0,oo) x R" x R m , where a±, a 2 are class /Coo functions, p is a class JC function, and W(x) is a 
continuous positive definite function on R ra . Then, the system \2.1\) is input-to-state stable (ISS). 


Remark 1. Note that other equivalent definitions for ISS have been given in 1231 pp. 1974-1975]. For instance, 
Theorem \2.1I holds with all the assumptions are the same except that the inequality 12.21) is replaced by 

+ (*’ x ’- -mind+ mni) 

where p £ /C^ H C 1 and O £ /C^. 


3 The case of nonlinear system model with state dependent uncertainties 

We consider here affine uncertain nonlinear systems of the form: 

x = f(x) + Af(x) + g(x)u, x(0)=x 0 

{0.6) 

y = h(x) 

where x £ R", u £ R p , y £ R m (p > m), represent respectively the state, the input and the controlled output 
vectors, xq is a given initial condition, A f(x) is a vector field representing additive model uncertainties. The 
vector fields /, A/, columns of g and function h satisfy the following assumptions. 

Assumption 1. The function f : R™ —> R" and the columns of g : R n —> R p are C°° vector fields on a bounded 
set X o/M" and h : R n —> R m is a C°° vector on X. The vector field A fix) is C 1 on X. 

Assumption 2. System \ 4 ■ has a well-defined ( vector ) relative degree {ri, r 2 , • • • , r m } at each point x° £ X, 
and the system is linearizable, i.e. JfoLi r i = n - 

Assumption 3. The desired output trajectories yid (1 < i 24 rn) are smooth functions of time, relating desired 
initial points yid{0 ) at t = 0 to desired final points yiditf) at t = tf. 

3.1 Control objectives Our objective is to design a state feedback adaptive controller so that the tracking 
error is uniformly bounded, whereas the tracking upper-bound can be made smaller over the ES learning iterations. 




We stress here that the goal of the ES is not stabilization but rather performance optimization, i.e. estimating 
online the uncertain part of the model and thus improving the performance of the overall controller. To achieve 
this control objective, we proceed as follows. First, we design a robust controller which can guarantee the input- 
to-state stability (ISS) of the tracking error dynamics w.r.t the estimation errors input. Then, we combine this 
controller with a model-free extremum-seeking algorithm to iteratively estimate the uncertain parameters, to 
optimize online a desired performance cost function. 

3.2 Nominal Controller Design Under Assumption 2 and nominal conditions, i.e. when A f(x) = 0, system 
(14.28|1 can be written as 


(3.4) 

y {r \t) = bm)+mt))<t ), 

where 

y {r \t ) = [y[ ri \t), yi r2) {t), ••• , ylf m) {t)] T 

(3.5) 

m = im •••, r(t)] T 

C(t) = [ln(t), ••• , 2/i n_1) W], 1 < * < m 


The functions 6(£), A(£) can be written as functions of /, g and h, and A(£) is non-singular in X, where X is the 
image of the set of X by the diffeomorphism x —> £ between the states of system (|4.28l) and the linearized model 

At this point, we introduce one more assumption on system (14.281) . 

Assumption 4. The additive uncertainties A f(x) in l[4-28\ ) appear as additive uncertainties in the input-output 
linearized model \4-29ty - \4-30\ ) as follows (see also \1T( ) 

(3.6) y (r \t) = &(£(*)) + A(£(t))u(t) + A6(£(i)), 
where A6(£(t)) is C 1 on X. 

If we consider the nominal model (14. 291) first, then we can define a virtual input vector v(t) as 

(3.7) v(t) = &(£(*)) + A(£{t))u(t). 

Combining (14.291) and (14.321) . we can obtain the following input-output mapping 

(3.8) 2/ (r) C t) = v(t). 

Based on the linear system (14.331) . it is straightforward to apply a stabilizing controller for the nominal system 
(14.291) as 


( 3 . 9 ) 


Un= A ^ [v s (t,Z) - 6(C)], 

























where v s is a m x 1 vector and the *-th (1 < * < in) element v S i is given by 

(3-10) v si = - K i r .(y ( T i ~ 1) - y^ 4_1) )- K\{ yi - y id ). 

Denote the tracking error as ei(t) = yi{t) — yid(t), we obtain the following tracking error dynamics 
(3.11) ej r<) (t) + (*) + •••+ K\ei(t) = 0, 

where i £ { 1,2, • • • , m}. By selecting the gains if) where ie{ 1,2, • • • , m} and j £ {1, 2, • • • , r,}, we can 
obtain global asymptotic stability of the tracking errors ei{t). To formalize this condition, we make the following 
assumption. 


Assumption 5. There exists a non-empty set A where K'j £ A such that the polynomials in H 4 - 36 \ ) are Hurwitz, 
where i £ {1, 2, • • • , m} and j £ {1, 2, • • • , ri}. 

To this end, we define z = [z 1 , z 2 , ■■■ , z m ] T , where z* = [ej, e'*, , e- ri_1) ] and i £ {1, 2, - - - , m}. 

Then, from (14.361) . we can obtain 

z = Az, 


where A £ R nxn is a diagonal block matrix given by 
(3.12) A = diagjAi, A 2 , 

and Aj (1 < * < m ) is a r* x r l matrix given by 

1 


A, = 


~K\ -K' 2 


1 




1 

-Ki 


As discussed above, the gains if) can be chosen so that the matrix A is Hurwitz. Thus, there exists a positive 
definite matrix P > 0 such that (see e.g. W) 


(3.13) 


A t P + PA = -I. 


3.3 Robust Controller Design We now consider the uncertain model (14.281) . i.e. when Af(x) ^ 0. The 
corresponding linearized model is given by (14.311) where A b(£(t)) ^ 0. The global asymptotic stability of the 
error dynamics (14.361) cannot be guaranteed anymore due to the additive uncertainty A 6 (£(f)). We use Lyapunov 
reconstruction techniques to design a new controller so that the tracking error is guaranteed to be bounded given 
that the estimate error of A 6 (£(i)) is bounded. The new controller for the uncertain model (14.311) is defined as 


(3.14) 


Uf 


U n + U r , 
















where the nominal controller u n is given by (|4.34J) and the robust controller u r will be given later on based on 
particular forms of the uncertainty A&(£(£)). By using the controller (|4.39l) . from (14.311) we obtain 


y {r) (t ) = b(£(t)) + A(£(t))u f +Ab(£(t)) 

= b(£(t)) + A(£(t))u n + A(£(t))u r + A b(£(t)) 
(3.15) = v a {t,£)+A(£(t))u r + Ab(£(t)), 


where (14.401) holds from ()4.34l) . Thus, we have 

e^it) + + ■■■ + K[ ei {t) 

(3.16) = A(£(t))u r + A b(£(t)) 


Further, the dynamics for z is given by 

(3.17) z = Az + BS, 

where A is defined in (14.371) . <5 is a m x 1 vector given by 


(3.18) 


S = A(£(t))u r + Ab{£{t)), 


and the matrix B € K nxm is given by 


(3.19) 


Bi 

B 2 


with Bi (1 < i < m ) given by a x m matrix such that 




1 if l = Ti and q = i 

0 otherwise 


If we apply V(z) = z T Pz as a Lyapunov function for the dynamics (14.411) . where P is the solution of the Lyapunov 
equation (|4.38l) . then we obtain 


(3.20) 


V(t) 


dV . 

dz Z 

z T (A T P + PA)z + 2z t PB5 
- \\ z \\ 2 + 2z t PB5 1 


where S given by (14.421) depends on the robust controller u r . 

Next, we will design the controller u r based on the particular forms of the uncertainties that appear in (14.311) . 
i.e. A6(£(f)). For notational convenience, the unknown parameter vector/matrix is denoted by A and the estimate 
for the unknowns is denoted by A (t). Further, the estimation error vector/matrix is given by e&(t) = A — A (i), 
where the dimensions of A (and in turn, A (t) and et\(t) will be clear from the context. 
























3.4 The case of bounded state-dependent uncertainties We consider the case where the unknown 
||Afo(£(i))|| is upper bounded by a function of the state £(t), i.e. 

(3-21) ||Ab(£(i))|| < ||A||||L(0||, 

where A £ M mxm is constant, and L(£) is a known bounded state function. Assume, for now, that we can obtain 
the estimate of A(z, j), which may be time-varying and is denoted by A (i,j), for i. j = 1,2,..., to. Let Aft) be 
the matrix with the element AWe use the following robust controller 

u r = - A-\OB T Pz\\m \\ 2 

(3.22) -A-^OIIAWHIIL^IIsign^P^). 

The closed-loop error dynamics can be written in the form of 

(3.23) i = f(t,z,e A ), 
where e A (t) is the system input and z(t) is the system state. 

Theorem 3.1. Consider the system {4-281 ), under Assumptions 1-5 and the assumption that A b(£(t)) satisfies 
14-4 7P ; w ith the feedback controller \4- 39\) , where u n is given by f 4-34\) and u r is given by f 4-481 ). Then, the 
closed-loop system 1 4-4^ ISS from the estimation errors input e A (t) £ R mxm to the tracking errors state 
z(t) £ K n . 

Proof. By substitution (14.481) into (14.421) . we obtain S = — B T Pz\\L(f)\\ 2 — ||A(t)||||L(£)||sign(.B T P,z) + A b(£(t)). 
We consider V(z) = z T Pz as a Lyapunov function for the error dynamics (14.41|) . where P > 0 is a solution of 
(14.381) . We can derive that 

(3.24) A min (P)||z || 2 < V(z) < A max (P)||c|| 2 , 

where A m ; n (P) > 0, A max (P) > 0 denote respectively the minimum and the maximum eigenvalues of the matrix 
P. Then, from (14.441) . we obtain 

V = -\\z\\ 2 + 2z T PBAb(f(t)) 

2 ||z t PP|| 2 ||Z /(£)|| 2 - 2||z T PP|| 1 ||A(t)|||| J L(0||. 

Since ||z T PP|| < ||z T PP||i, we have 


V < - \\z\\ 2 + 2z T PBAb(£(t)) 

- 2||z t PP|| 2 ||L(0 || 2 - 2||z T PP||||A(t)||||P(0||. 
























Then based on the assumption (14.471) and the fact that z T PBAb(f(t)) < ||z T P.B||||A6(£(f))||, we obtain 

v < -\\z\\ 2 + 2\\z T PB\\\\A\\\\L(m 

- 2||^PP|| 2 || j L( 0|| 2 - 2||z r PP||||A(f)||||L(£)|| 

= -|| 2 || 2 -2||z t PP|| 2 ||P(^)|| 2 

+ 2||z T PP||||P(£)||(||A|| — ||A(t)||). 

Because ||A|| — ||A(t)|| < ||||, we obtain 

V < -\\z\\ 2 -2\\z T PB\\ 2 \\m\\ 2 

+ 2||z T PP||||L(£)||||e A ||. 

Further, we can obtain 

V < -|NI| 2 -2(||z T PP||||P(0||-i||e A ||) 2 + i||e A || 2 

< - Ikll 2 -F |l|eA|| 2 . 

Thus, we have the following relation 

V<~\\z\\ 2 , V||z|| > ||e A || > 0, 

Then from (13.241) . we obtain that system (14.491) is ISS from input e A to state z as guaranteed by Theorem 12. II 

3.5 Multi-parametric ES-based uncertainties estimation Let us define now the following cost function 
(3.25) J(A) = F(z( A)) 

where F : R ra —> R, P(0) = 0, F(z) >0 for 0. We need the following assumptions on J. 

Assumption 6. The cost function J has a local minimum at A* = A. 

Assumption 7. The initial error e A (fo) is sufficiently small, i.e. the original parameter estimate vector or matrix 
A are close enough to the actual parameter vector or matrix A. 

Assumption 8. The cost function J is analytic and its variation with respect to the uncertain parameters is 
bounded in the neighborhood of A*, i.e. ||^i(A)|| < £ 2 ) £2 > 0, A 6 V(A*) ; where V(A*) denotes a compact 
neighborhood of A*. 

Remark 2. Assumption \14\ simply states that the cost function J has at least a local minimum at the true values 
of the uncertain parameters. 


Remark 3. Assumption [7] indicates that out results will be of local nature, i.e. our analysis holds in a small 
neighborhood of the actual values of the uncertain parameters. 








We can now state the following result. 


Lemma 3.1. Consider the system \4-38\ ) with the cost function H4-53\ ), under Assumptions 1-8 and the assumption 
that A 6 (£(f)) = [A 1; .. . , A m ] T , with the feedback controller where u n is given by J 4-341 ) and u r is given 

by h4-48j )i an 3 A (t) is estimated through the MES algorithm 

7T ^ 

Xi = at sin(wjf + —) J(A) 

(3.26) A i(t) = Xi + aism(u)it-^), ig{l,2,...,m} 

with uji ^ u>j, Ui + LOj ^ u>k, i,j, k € {1,2,..., m}, and uji > w*, V i £ {1, 2,..., to}, with uj* large enough, ensures 
that the norm of the error vector z(t ) admits the following bound 

IkWII </3(lk(0)|M) + 7(^(l|eA(0)|M) + ||e A || max ) 

where ||e A ||max = ^ + \/Xa=i a i > 6 > e A (0) S V e , w 0 = max l6 { 1:2r .., m } w i; /3 € ICC, ft e ICC and 7 C 1C. 

Proof. Based on Theorem 14.21 we know that the tracking error dynamics (14.4911 is ISS from the input e A (f) to 
the state z(t). Thus, by Definition 12.11 there exist a class ICC function /3 and a class K, function 7 such that for 
any initial state ,s( 0 ), any bounded input e A (f) and any t > 0, 


(3.27) IkWII < £(l|z(0)IM) +7( sup ||e A (r)||). 

0 <r<t 

Now, we need to evaluate the bound on the estimation vector A (£), to do so we use the results presented 
in m- First, based on Assumption [15l the cost function is locally Lipschitz, i.e. there exists rji > 0 such 
that |J(Ai) — J(A 2 )| < 771 1|Ai — A 2 II, for all Ai,A 2 € V(A*). Furthermore, since J is analytic, it can be 
approximated locally in V(A*) by a quadratic function, e.g. Taylor series up to the second order. Based on this 
and on Assumptions IT4l and 171 we can obtain the following bound ([10j p. 436-437] Jl4l) 

||e A (f)|| - ||d(i)|| < ||e A (i) - d(i)|| < /3(||e A (0), t||) + A 

w 0 

where /3 £ ICC , £1 > 0, f > 0, wo = maxj e {iw^, and d{t) = [ai sin(wit + -|),..., a m sin(a; rn t + f )] T . We can 
further obtain that 


||e A (t)||< /3(||e A (0),i||) + — + ||d(t)|| 

wo 


< /3(||e A (0),t||) + — + 
w 0 


\ J2 a i 

\ i=1 


Together with (13.271) yields the desired result. 


4 The case of nonlinear system model with time dependent uncertainties 

We consider here affine uncertain nonlinear systems of the form: 

x = f{x) + Af(t,x) + g(x)u 
V = h{x), 


(4.28) 



















where x € R", u £ R p , y £ R m ( p > m), represent respectively the state, the input and the controlled output 
vectors, A f(t,x) is a vector field representing additive model uncertainties. The vector fields /, A/, columns of 
g and function h satisfy the following assumptions. 

Assumption 9. The function f : R ra — > R" and the columns of g : R n —> R p are C°° vector fields on a bounded 
set X o/M" and h : R n — > R m is a C°° vector on X. The vector field A f is C 1 on X. 

Assumption 10. System . 28\) has a well-defined (vector) relative degree {r\, r%, ••• , r m } at each point 
x° £ X, and the system is linearizable, i.e. Y^iLi A = n - 

Assumption 11. The desired output trajectories ytd (1 < i < m) are smooth functions of time, relating desired 
initial points yid{ 0 ) at t = 0 to desired final points yiditf) at t = tf. 

4.1 Control objectives Our objective is to design a state feedback controller u(x) so that for the uncertain 
nonlinear model (14.281) the tracking error is uniformly bounded. We stress here that the goal of parameters tuning 
is not for stabilization but for performance optimization. To achieve the control objective, we proceed as follows. 
First, we design a robust controller which can guarantee the input-to-state stability (ISS) of the tracking error 
dynamics w.r.t the estimation errors input. Then, we combine this controller with a model-free extremum-seeking 
algorithm to iteratively tune the uncertain parameters, to optimize online a desired performance cost function. 

4.2 Nominal Controller Design Under Assumption 2 and nominal conditions, i.e. when A f{t,x) = 0, 
system (|4.28l) can be written as, e.g. [lj 

( 4 . 29 ) i r \t) = bm)+mt)Mt), 

where 

y (r \t) = [y^it), y^\t), ••• , y^\t)} T 

( 4 . 30 ) m •••, r(t)] T 

C(t) = ••• » J/i ri_1) ( t )]> l<i<m 

The functions fe(£), A(£) can be written as functions of /, g and h , and A(£) is non-singular in X, where X is the 
image of the set of X by the diffeomorphism x —> £ between the states of system (14.281) and the linearized model 
(l4~29l) . 

At this point, we introduce one more assumption on system (14.281) . 

Assumption 12. The additive uncertainties A f(t,x) in ff . 28\) appear as additive uncertainties in the input- 
output linearized model 30\j as follows (see also fll^ ) 

(4.31) y {r \t) = b(£(t)) + A(£(t))u(t)+Ab(t,t;(t)), 


where Ab(t, £,{t)) is C 1 on X. 




















If we consider the nominal model (14.2911 first, then we can define a virtual input vector v(t) as 


(4.32) v(t) = b{£{t)) +A(£(t))u{t). 

Combining (14.291) and (14.321) . we can obtain the following input-output mapping 

(4.33) y^(t) = v(t). 

Based on the linear system (14.331) . it is straightforward to apply a stabilizing controller for the nominal system 
(I4.29P as 

(4.34) u n = A _1 (£) [v s (t, £) - 6(0], 
where v s is a to x 1 vector and the *-th (1 < i < m ) element v S i is given by 

( 4 -35) v si = y^ i] - K l r .(y ( f i ~ 1) - y^ i_1) )- K{{ Vl - y id ). 

Denote the tracking error as ei(t) = yi{t) — yi d {t ), we obtain the following tracking error dynamics 
(4.36) e( ri) (t) + K i r .e^ ri ~ 1 ' > (f) + • • • + K[ ei {t) = 0, 

where z€{1,2, • • • , m}. By selecting the gains K l - where * G {1, 2, • • • , to} and j G {1,2, • • • , r*}, we can 
obtain global asymptotic stability of the tracking errors ei(t). To formalize this condition, we make the following 
assumption. 


Assumption 13. There exists a non-empty set A where KJ- G A such that the polynomials in \4-36\j are Hurwitz, 
where i G {1, 2, • • • , to} and j G {1, 2, • ■ • , r .-,}. 

To this end, we define z = [z 1 , z 2 , ••• , z m ] T , where z‘ = [e*, e}, ••• , e( r,_1) ] and i G {1, 2, - - - , to}. 
Then, from (|4.36L we can obtain 

i = Az, 


where A G R” XTl is a diagonal block matrix given by 


(4.37) 


A = diag{Ai, A 2 , , A m }, 


and Ai (1 < * < m) is a r* x r, matrix given by 


A,; = 


: 1 
_ -a; . -K* t _ 

As discussed above, the gains K‘ can be chosen so that the matrix A is Hurwitz. Thus, there exists a positive 
definite matrix P > 0 such that (see e.g. [IB]) 

A t P + PA = 


(4.38) 


-I. 


















4.3 Robust Controller Design We now consider the uncertain model (|4.28l) . i.e. when A f(t,x) ^ 0. The 
corresponding linearized model is given by (14.3111 where A6(i,£(f)) ^ 0. The global asymptotic stability of the 
error dynamics (14.361) cannot be guaranteed anymore due to the additive uncertainty A6(t, £(f)). We use Lyapunov 
reconstruction techniques to design a new controller so that the tracking error is guaranteed to be bounded. The 
new controller for the uncertain model (14.311) is defined as 

(4.39) Uf = u n + u r , 

where the nominal controller u n is given by (14.34[) and the robust controller u r will be given later on based on 
particular forms of the uncertainty A&(f, £(f)). By using the controller (14.391) . from (14.311) we obtain 

y {r \t) = &(£(*)) + A (£(t))u f + Ab(t,£(t)) 

= &(£(*)) + A (f W)«n + A((,(t))u r + A bit, £(*)) 

(4.40) = v s (t,£)+A(£(t))u r +Ab(t,£(t)), 


Further, the dynamics for z is given by 

(4.41) z = Az + BS, 

where A is defined in (14.371) . S is a m x 1 vector given by 


(4.42) 


S = A(£(t))u r + Ab{t,£(t)), 


and the matrix B € K nxm is given by 


(4.43) 


B i 

B 2 


with Bi (1 < i < m) given by a q x m matrix such that 




1 if l = Ti and q = i 

0 otherwise 


If we apply V(z) = z T Pz as a Lyapunov function for the dynamics (14.411) . where P is the solution of the Lyapunov 
equation (14.381) . then we obtain 


V(t) 


dV . 

!h z 

z T (A T P + PA)z + 2z t PB5 
- \\ z \\ 2 + 2z t PB5 1 


(4.44) 
























where S given by (14.421) depends on the robust controller u r . 


Next, we will design the controller u r based on the particular forms of the uncertainties that appear in 
(14.311) . i.e. Ab(t, £(t)). For notational convenience, the unknown parameter vector/matrix (which may be time- 
varying) is denoted by A(f) and the estimate for the unknowns is denoted by A (t). Further, the estimation error 
vector/matrix is given by e A (t) = A (t) — A (t). The dimensions of A (and in turn, A and e A ) will be clear from 
the context. 

4.4 Case 1 : State-Independent Uncertainties We consider the case when A b(t, £(f)) is simply A(f), where 
A (t) = [Ai(t),..., A m{t)] T . Assume that we can obtain the estimate (e.g. by ES) of the unknown parameters 
Ai(f), which may be time-varying and is denoted by A i(t), for i = 1,2,..., m. Let A(i) = [Ai(t),..., A m (f)] T . 
We use the following robust controller 

(4.45) Ur = -A- 1 (£)(B T Pz + A(t)). 

The closed-loop error dynamics can be written as 

(4.46) i = f(z,e A ), 

where e A (t) is the input to the system, z(t) represents the system state and / is given by 

f{z,e A ) = (A- BB T P)z + Be A . 


Theorem 4.1. Consider the system /j4-28\ ), under Assumptions 1-5 and the assumption that A5(f,£(f)) = 
[Ai(f),..., A m (t)] T , with the feedback controller \4-39j) , where u n is given by 14-34 1 ) and u r is given by \4-45\) - 
Then, the closed-loop system f4-4bj ) is ISS from the estimation errors input e A (t) £ M m to the tracking errors 
state z(t ) £ M”. 

4.5 Case2: State-Dependent Uncertainties We consider the second case when ||A6(i,£(f))|| is upper 
bounded by a function of the state £(i), i.e. 

(4-47) l|A&Mi))|| < l|A(t)||||L(0||, 

where A(t) £ K. mxm and L(£) is a known bounded function. Assume that we can obtain the estimate (e.g. by ES) 
for A (i,j), which may be time-varying and is denoted by A (i,j), for i,j = 1,2,... ,m. Let A(t) be the matrix 
with the element A (i,j). We use the following robust controller 

u r = -A-\H)B T Pz\m)f 

- A-\Omt)\\\\m\\Agn{B T Pz). 


(4.48) 
















Similar to the previous case, the closed-loop error dynamics can be written in the form of 

(4.49) i = f(t,z,e A ), 
where e A (t) is the system input and z(t) is the system state. 

Theorem 4.2. Consider the system \4.28fy , under Assumptions 1-5 and the assumption that Ab(t,£(t)) satisfies 
14.4?\ ), with the feedback controller \4-39j) , where u n is given by \4■ 3 41 ) and u r is given by Then, the 

closed-loop system 1 4-49\ ) is ISS from the estimation errors input e A ft) £ M mxm to the tracking errors state 
z(t) £ R". 

4.6 Case 3: Sum of a State-dependent Term and a Time-dependent Term We consider the third case 
when A b(t,£(t)) is composed of a state-dependent term and a time-dependent term, i.e. 

(4.50) A . 6 (t,£(t)) = A(i)(Q(£) + 77 (f)), 

where Aft) £ R mxm , Q(f) is a known bounded function, the vector 77 (f) is unknown but the upper bound for 
||? 7 (f)|| is known to be C\, i.e. ||r 7 (f)|| < Ci. Assume that we can obtain the estimate (e.g. by ES) for A (i,j), 
which may be time-varying and is denoted by A (i,j), for i,j = 1,2, ... ,m. Let Aft) be the matrix with the 
element A (i,j) that locates at the i-th row and j-th column. We use the following robust controller 

u r = -A-^^PzWQ^lf + A^xQiO 

(4.51) + ||A(t)||Cisign(B T Pz) + B T PzCf}. 

Similar to the previous two cases, the closed-loop error dynamics can be written in the following form 

(4.52) i = F(t,z,e a), 
where e A (t) is the system input and zft) is the system state. 

Theorem 4.3. Consider the system j4-28l ), under Assumptions 1-5 and the assumption that Ab(t,£ft)) satisfies 
1 4-50\ ), with the feedback controller \f.3Efy , where u n is given by \4-34 1 ) and u r is given by j^.51[ j. Then, the 
closed-loop system 1 4-52\ ) is ISS from the estimation errors input e A ft) £ errors state 

zft) £ K n . 

4.7 Multi-parametric ES-based Adaptation Let us now define the following cost function 

(4.53) J(A, t) = F(z(A), t) 

where F :R n x M+ —> M + , F(0, t) = 0, F(z, t) > 0 for z 0. We need the following assumptions on J. 
Assumption 14. The cost function J has a local minimum at A* ft) = A(t). 


























Assumption 15. | aj | < pj, for any t g K + and any A g R p . 


Remark 4. Assumption \14\ simply states that the cost function J has at least a local minimum at the true values 
of the uncertain parameters. 

We can now present the following result for Case 1, i.e. the case that is studied in Section [TTl 

Lemma 4.1. Consider the system 1 4-41\ ) with the cost function \4-53\) , under Assumptions |1^[I15I and the 
assumption that Aft) = [Ai(t),..., A m (t)] T , with the feedback controller h4-39\ ), where u n is given by 1 4-341 ) 
and u r is given by \4-45\ ), and Aft) is estimated through the MES algorithm 

(4.54) A i = aiy/uJiCOs(uJit) — kiy/upsm(u;it)J(A,t), 

where i g {1, 2,.. . ,p}, a* > 0, ki > 0, uii ^ ujj, and uji > uj* , with uj* large enough, ensures that the norm of the 
tracking error admits the following bound 


lk(i)|| < P(\\zft 0 )\\,t) + 'y( sup ||eA(r)||) 

0<r<£ 

where /? g ICC, 7 g K and ||e^.|| satisfies: 

1. ( —, d)-Uniform Stability: For every C 2 g (d, 00 ), there exists C\ g (0,oo) and 6 j > 0 such that for all t 0 g R 
and for all ca(0) g M m with ||ca ( 0 ) || < Ci and for all uj > ui , 

||eA(t,e A (0))|| < c 2 , Vt g [t 0 ,00) 

2. (i ,d)-Uniform Ultimate Boundedness: For every Ci g (0,oo), there exists c 2 g (d, 00 ) and uj > 0 such that 
for all to g R. and for all ca(0) g with ||ca ( 0 ) || < Ci and for all uj > uj, 

||e A (t, e A (0))|| < c 2 , Vt g [t 0 , 00 ) 


3. (i ,d)-Global Uniform Attractivity: For all ci, c 2 g (d, 00 ) there exists T g [0, 00 ) and uj > 0 such that for 
all t 0 g R and for all ca(0) g K m with || eA ( 0 )|| < ci and for all uj > uj, 


||e A (t, e A (0))|| < c 2 , Vt g [t 0 + T, 00 ) 


Similar bounds can be derived for the two remaining cases but are omitted here because of space constraints. 

5 Mechatronic Example 

We consider here a two-link robot manipulator example. The dynamics for the manipulator in the nominal case, 
is given by (see e.g. [ 20) 1 


(5.55) 


H(q)q + C(q,q)q + G(q) 













Tabic 1: System Parameters for the manipulator example. 


Parameter 

Value 

I‘2 

T 2 i k 9 ■ to 2 ] 

mi 

10 [kg] 

m 2 

5 [kg] 

4 

1 [to] 

£2 

1 [to] 

£ C1 

0.5 [to] 

£c 2 

0.5 [to] 

h 

T§ i k 9 ' to 2 ] 

9 

9.8 [to/s 2 ] 


where q = [gi, q 2 ] T denotes the two joint angles and r = [ti, t 2 ] t denotes the two joint torques. The matrix H is 
assumed to be non-singular and is given by 

r H n H 12 
H 2 \ h 22 


H 4 


where 

H u 

(5.56) 

H\ 2 


h 2 1 


H 22 


Hu = +Ji +m 2 [i\ + £ 2 C2 + 2£i 4 2 cos(g 2 )] + h 

£ 


4 


C{q,q) = 


(5.57) 


The matrix G(g, q) is given by 

— hq 2 —hqi — hq 2 
hqi 0 

where ft. = m 2 £\£ C2 sin(g 2 ). The vector G = [Gi, G 2 ] T is given by 

Gi = mi£ Cl g cos(gi) + m 2 g[£ 2 cos(gi + g 2 ) + 4 cos(gi)] 

G 2 = m 2 £ C2 g cos(gi + q 2 ) 

In our simulations, we assume that the parameters take values according to [20] summarized in Table |T| The 
system dynamics (]5.55l) can be rewritten as 

(5.58) q = H~ 1 (q)r — H~ 1 {q) [C(q, q)q + G(q)} , 

Thus, the nominal controller is given by 


(5.59) 


r n = [C{q,q)q + G(q)} 

+ H(q) [q d — K 2 (q — q d ) - K^q - q d )} 



















where qd = [qid, Q 2 d] T , denotes the desired trajectory and the feedback gains K\ > 0, AT 2 > 0, are chosen such 
that the tracking error will go to zero asymptotically. For simplicity, we use the feedback gains AT] = 1 in <j4.35(l 
for i = 1,2 and j = 1, 2 in our simulations. The reference trajectory is given by the following function from the 
initial time to = 0 to the final time tf , where 


qid(t) 


1 

1 + exp (—t) ’ 


1,2 


5.1 State dependent uncertainties Now we introduce an uncertain term to the nonlinear model (15.551) . In 
particular, we assume that there exist additive uncertainties in the model (15.581) . i.e. 


(5.60) 


q = H-\q)r - H~\q ) [C(q, q)q + G(q )] + A b(q). 


We assume additive uncertainties on the gravity vector, such that 


(5.61) Ab(q) = A x G(q), 

so that we have ||A6(g)|| < ||A||||G(g)||. For simplicity, we assume that A is a diagonal matrix given by 
A = diag{Ai,A 2 }. The robust controller term r r is designed according to (14.481) . where H = A _1 , L = G, 
and finally the two unknown parameters Ai and A 2 are estimated by the MES, as shown in the next section. 


5.2 MES Based uncertainties estimation First, we choose the following performance cost function 


(5.62) 


J = Q i Jo f (q - qd) T {q - qd)dt 
+Q2 fo f (q - qd) T (q - qd) dt, 


where Q 1 > 0 and Q 2 > 0 denote the weighting parameters. Then, the two unknown parameters Ai and A 2 are 
estimated by the MES (which is a discrete version of (15.261) 1 


(5.63) 


M*+l) = x,{k) + u,t,em(u l t f k+ -)J 

^ 7T 

Aj(fc + 1) = Xi(k + 1) + aisinfaitfk - -), * = 1,2 


where k = 0, 1, 2, - - ■ denotes the iteration index, Xi and Aj (i = 1,2) start from zero initial conditions. We 
simulate the system with Ai = —1 and A 2 = —3. The parameters that were used in the cost function (15.651) and 
the MES (15.671) are summarized in Table [2j As shown in Fig. 15.21 the ISS-based controller combined with ES 
greatly improves the tracking performance. Fig. [2] shows that the cost function starts at an initial value around 6 
and decreases below 0.5 within 100 iterations and the value of the cost function is decreasing over the iterations. 
Moreover, the estimate of the unknown parameters converge to a neighborhood of the true parameter values, as 
shown in Fig. [3] 


5.3 Time dependent uncertainties Now we assume that there exist additive time-dependent uncertainties 
in the model (15.581) . i.e. 


(5.64) 


q = H~\q)r - H~\q ) [C(q, q)q + G(q)} + A b(q, t). 



















Table 2: Parameters used in MES 


Qi 

Q 2 

ai 

a 2 

Ul 

W 2 

tf 

5 

5 

0.05 

0.04 

7.4 

7.5 

4 
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-with learning 

■ without learning 




time 


Figure 1: Obtained trajectories vs. Reference Trajectories for qi [rad] (in the top plot) and qi [rad/sec] (in 
bottom plot) 


cost function 



Figure 2: The cost function vs. the number of iterations 
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Figure 3: Estimate of parameter Ai (in the top plot) and parameter A 2 (in the bottom plot) 





































We will illustrate our approach for the uncertain model (15.641) . In the following, we consider the cost function 

(5.65) J = Qi [ \\q-qd\\ 2 dt + Q 2 f ||<j - <j d || 2 dt, 

Jo Jo 

where Q 1 > 0 and Q 2 > 0 denote the weighting parameters. 

5.4 Time-varying MES Based Adaptation Due to space limitation, we report hereafter only the case 
presented in Section T4.41 when A b(q,t) is simply a time-varying vector A(i), where A(t) = [Ai(i), A 2 (t)] T . 

However, we underline that we have successfully tested the remaining cases and that all the results will be 
reported in a longer journal version of this work. 

Here the controller is designed according to Theorem 14.11 and the two unknowns Ar(t) and A 2 (t) are identified 
by the MES (14.541) such that the cost function J in (15.651) is minimized. 

We simulate the system with 

Ai (t) = 1 — 0.14sin(0.01f) 

(5.66) A 2 {t) = 1 — 0.12cos(0.01t). 

The estimate of these two parameters A, (i = 1,2) are computed using a discrete version of (14.541) . given by 

A i(k + 1) = A i(k) + tf(ais/uicos(uitfk) 

(5.67) — Kiy/7JiS\n(uJitfk) J), i = 1,2 

where k = 0,1, 2, • - • denotes the iteration index and A* (i = 1,2) start from zero initial conditions. The 
parameters used in the cost function (15.651) and the MES (15.671) are summarized in Table [3] For more details 
about the tuning of the parameters in the MES, we refer the reader to [5j. However, we underline here that the 
frequencies u >i and oj 2 have been selected high enough to ensure efficient exploration on the search space and to 
ensure convergence and that the amplitudes at and re,; (i = 1,2) of the dither signals have been selected such 
that the search is fast enough. The uncertain parameter vector A is updated for each cycle, i.e. at the end of 
each cycle at t = tf , the cost function J is updated, and the new estimate of the parameters is computed for the 
next cycle. The purpose of using MES along with the ISS-based controller is to improve the performance of the 
controller by better estimating the unknown parameters over many cycles, hence decreasing the estimation errors 
over time to provide better trajectory tracking. As shown in Fig. [2 the ISS-based controller combined with ES 
greatly improves the tracking performance. Fig. [6] show that the cost function starts at an initial value around 
7 and decreases below 1 within 20 iterations. Moreover, the estimate of the unknown parameters converge to a 
neighborhood of the true parameter trajectories within 100 iterations, as shown in Fig. [7] 

6 Conclusion 

In this paper, we studied the problem of extremum seeking-based indirect adaptive control for nonlinear systems 
affine in the control with bounded additive state-dependent uncertainties. We have proposed a robust controller 














Table 3: parameters used for case 1. 
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Figure 4: Case 1: Obtained trajectories vs. reference trajectories for qi (in the top plot) and q\ (in the bottom 
plot). 
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Figure 5: Case 1: The cost function vs. the number of iterations. 

which renders the feedback dynamics ISS w.r.t the parameter estimation errors. Then we have combined the 
ISS feedback controller with a model-free ES algorithm to obtain a learning-based adaptive controller, where the 
ES is used to estimate the uncertain part of the model. We have presented the stability proof of this controller 
and have shown a detailed application of this approach on a two-link robot manipulator example. Future work 
will deal with considering controllers under input constraints, using different ES/learning algorithms with less 
restrictive tuning conditions, and comparing the obtained controllers to some available classical nonlinear adaptive 
controllers. 
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